Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

salonichf5 · 2025-11-18T22:45:42Z

Proposed changes

Write a clear and concise description that helps reviewers understand the purpose and impact of your changes. Use the
following format:

Problem: Users want to be able to specify session persistence for their upstreams.

Solution: Add support for session persistence using sticky cookie directives which is only available for NGINX Plus users

Testing: Manual testing

Errors

Spec validation

The HTTPRoute "coffee" is invalid: 
* spec.rules[0].sessionPersistence.sessionName: Too long: may not be more than 128 bytes
* <nil>: Invalid value: null: some validation rules were not checked because the object was invalid; correct the existing errors to complete validation


    sessionPersistence:
      sessionName: "coffee-shop-session-with-very-long-name-for-testing-session-affinity-functionality-in-kubernetes-gateway-api-implementation-v1-hello"

The HTTPRoute "tea" is invalid: spec.rules[0].sessionPersistence.absoluteTimeout: Invalid value: "99999d": spec.rules[0].sessionPersistence.absoluteTimeout in body should match '^([0-9]{1,5}(h|m|s|ms)){1,4}$'

The HTTPRoute "tea" is invalid: spec.rules[0].sessionPersistence.absoluteTimeout: Invalid value: "999d": spec.rules[0].sessionPersistence.absoluteTimeout in body should match '^([0-9]{1,5}(h|m|s|ms)){1,4}$'

httproute.gateway.networking.k8s.io/tea configured
The HTTPRoute "coffee" is invalid: spec.rules[0].sessionPersistence: Invalid value: "object": AbsoluteTimeout must be specified when cookie lifetimeType is Permanent

    sessionPersistence:
      sessionName: "coffee-cookie"
      cookieConfig:
        lifetimeType: Permanent

      Message:               The following unsupported parameters were ignored: [spec.rules[0].sessionPersistence.type: Unsupported value: "Header": supported values: "Cookie", spec.rules[0].sessionPersistence: Invalid value: "spec.rules[0].sessionPersistence": session persistence is ignored because there are errors in the configuration]

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Header
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

httproute.gateway.networking.k8s.io/tea configured
The HTTPRoute "coffee" is invalid: 
* spec.rules[0].sessionPersistence.type: Unsupported value: "cookie": supported values: "Cookie", "Header"
* <nil>: Invalid value: null: some validation rules were not checked because the object was invalid; correct the existing errors to complete validation

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

      Last Transition Time:  2025-11-18T01:28:53Z
      Message:               The following unsupported parameters were ignored: [spec.rules[0].sessionPersistence.idleTimeout: Invalid value: "300s": idleTimeout is not supported, spec.rules[0].sessionPersistence: Invalid value: "spec.rules[0].sessionPersistence": session persistence is ignored because there are errors in the configuration]

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      idleTimeout: 300s
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

NGF Errors

non-plus behaviour

      Last Transition Time:  2025-11-24T23:29:42Z
      Message:               All references are resolved
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-11-24T23:29:42Z
      Message:               The following unsupported parameters were ignored: spec.rules[0].sessionPersistence: Forbidden: SessionPersistence is only supported in NGINX Plus. This configuration will be ignored.
      Observed Generation:   1
      Reason:                UnsupportedField
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.nginx.org/nginx-gateway-controller
    Parent Ref:

non-experimental and non-plus behaviour

Parents:
    Conditions:
      Last Transition Time:  2025-11-24T23:32:30Z
      Message:               All references are resolved
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-11-24T23:32:30Z
      Message:               The following unsupported parameters were ignored: [spec.rules[0].sessionPersistence: Forbidden: SessionPersistence is only supported in NGINX Plus. This configuration will be ignored., spec.rules[0].sessionPersistence: Forbidden: SessionPersistence is only supported in experimental mode.]
      Observed Generation:   1
      Reason:                UnsupportedField
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.nginx.org/nginx-gateway-controller
    Parent Ref:
      Group:         gateway.networking.k8s.io
      Kind:          Gateway
      Name:          gateway
      Namespace:     default
      Section Name:  http
Events:              <none>

Configuration

HTTPRoutes

Case 1: path and expires specified

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent

upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie expires=1440m path=/coffee;

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;
}

Case 2 : no expiry, only path specified

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Session. —> no expiry



upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie path=/coffee;

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;

Regular expression with other path matches (no path set)

rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    - path:
        type: Exact
        value: /coffee/espresso
    - path:
        type: RegularExpression
        value: /coffee/[a-zA-Z0-9]+
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent



upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie; --.> no path

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;

Multiple matches with common prefix

 rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    - path:
        type: Exact
        value: /coffee/espresso
    - path:
        type: PathPrefix
        value: /coffee/tea
    - path:
        type: Exact
        value: /coffee/latte
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent



upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie path=/coffee;  --> common path
 
    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;

multiple matches with no common prefix

  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /1
    - path:
        type: Exact
        value: /2
    - path:
        type: PathPrefix
        value: /3
    - path:
        type: Exact
        value: /4
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent


upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie; --> no path

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;

GRPCRoutes (path is always empty for GRPCRoutes

Hostname matching

Hostname matching

  rules:
  - backendRefs:
    - name: grpc-infra-backend-v2
      port: 8080
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent

upstream default_grpc-infra-backend-v1_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v1_8080 1m;

    state /var/lib/nginx/state/default_grpc-infra-backend-v1_8080.conf;

}

upstream default_grpc-infra-backend-v2_8080_backend-v2_default_0 {
    random two least_conn;
    zone default_grpc-infra-backend-v2_8080_backend-v2_default_0 1m;
    sticky cookie coffee-cookie expires=1440m;

    state /var/lib/nginx/state/default_grpc-infra-backend-v2_8080_backend-v2_default_0.conf;

Exact Method matching

  rules:
  - matches:
    - method:
        service: helloworld.Greeter
        method: SayHello
    backendRefs:
    - name: grpc-infra-backend-v1
      port: 8080
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Session


upstream default_grpc-infra-backend-v1_8080_exact-matching_default_0 {
    random two least_conn;
    zone default_grpc-infra-backend-v1_8080_exact-matching_default_0 1m;
    sticky cookie coffee-cookie;

    state /var/lib/nginx/state/default_grpc-infra-backend-v1_8080_exact-matching_default_0.conf;

Traffic Splitting with HTTPRoutes

  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee-v1
    backendRefs:
    - name: coffee-v1-svc
      port: 80
      weight: 80
    - name: coffee-v2-svc
      port: 80
      weight: 20
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 10m
      cookieConfig:
        lifetimeType: Permanent

  - matches:
    - path:
        type: PathPrefix
        value: /coffee-diff-backend
    backendRefs:
    - name: coffee-v2-svc
      port: 80

  - matches:
    - path:
        type: PathPrefix
        value: /coffee-v2
    backendRefs:
    - name: coffee-v1-svc
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie-v2"
      type: Cookie
      absoluteTimeout: 24h
      cookieConfig:
        lifetimeType: Permanent

Upstreams generated

upstream default_coffee-v1-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee-v1;

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_0.conf;

}

upstream default_coffee-v1-svc_80_coffee_default_2 {
    random two least_conn;
    zone default_coffee-v1-svc_80_coffee_default_2 1m;
    sticky cookie coffee-cookie-v2 expires=1440m path=/coffee-v2;

    state /var/lib/nginx/state/default_coffee-v1-svc_80_coffee_default_2.conf;

}

upstream default_coffee-v2-svc_80 {
    random two least_conn;
    zone default_coffee-v2-svc_80 1m;

    state /var/lib/nginx/state/default_coffee-v2-svc_80.conf;

}

upstream default_coffee-v2-svc_80_coffee_default_0 {
    random two least_conn;
    zone default_coffee-v2-svc_80_coffee_default_0 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee-v1;

    state /var/lib/nginx/state/default_coffee-v2-svc_80_coffee_default_0.conf;

}

split_clients $request_id $group_default__coffee_rule0 {
    80.00% default_coffee-v1-svc_80_coffee_default_0;
    20.00% default_coffee-v2-svc_80_coffee_default_0;
}

location ^~ /coffee-v1/ {
  proxy_pass http://$group_default__coffee_rule0$request_uri;
}

Testing Session Persistence

Testing is done using a script by grabbing the cookie session id from the first request and using it in curl request to ensure each request goes to the same backend

HTTPRoutes (coffee -> sticky, tea -> regular)


upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee_80.conf; --> 3 endpoints
}

upstream default_tea_80 {
    random two least_conn;
    zone default_tea_80 1m;

    state /var/lib/nginx/state/default_tea_80.conf; --> 3 endpoints
}

curl -v --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
< Set-Cookie: coffee-cookie=2b13cc98f0224fca47885f32115ef236; expires=Tue, 18-Nov-25 22:17:55 GMT; max-age=600; path=/coffee


Summary (how many times each backend was chosen):
 100 10.244.0.111:8080

curl -v --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea
Summary (how many times each backend was chosen):
  32 10.244.0.108:8080
  38 10.244.0.112:8080
  30 10.244.0.113:8080

GRPCRoutes (backend v1 --> normal, backend v2 --> sticky)

Each have 3 endpoints


upstream default_grpc-infra-backend-v1_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v1_8080 1m;

    state /var/lib/nginx/state/default_grpc-infra-backend-v1_8080.conf;

}

upstream default_grpc-infra-backend-v2_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v2_8080 1m;
    sticky cookie grpc-v2-cookie expires=600s;

    state /var/lib/nginx/state/default_grpc-infra-backend-v2_8080.conf;

}

backend 1
grpcurl -v -plaintext -proto grpc.proto -authority bar.com -d '{"name": "bar server"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello
Handling connection for 8080

Resolved method descriptor:
// Sends a greeting
rpc SayHello ( .helloworld.HelloRequest ) returns ( .helloworld.HelloReply );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc
date: Tue, 18 Nov 2025 22:26:41 GMT
server: nginx

Response contents:
{
  "message": "Hello bar server"
}

Response trailers received:
(empty)
Sent 1 request and received 1 response

backend 2

rpcurl -v -plaintext -proto grpc.proto -authority foo.bar.com -d '{"name": "bar server"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello
Handling connection for 8080

Resolved method descriptor:
// Sends a greeting
rpc SayHello ( .helloworld.HelloRequest ) returns ( .helloworld.HelloReply );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc
date: Tue, 18 Nov 2025 22:27:28 GMT
server: nginx
set-cookie: grpc-v2-cookie=cc8db88effa3e2563d0b5b13054dc7cc; expires=Tue, 18-Nov-25 22:37:28 GMT; max-age=600

Response contents:
{
  "message": "Hello bar server"
}

Response trailers received:
(empty)
Sent 1 request and received 1 response
sa.choudhary@N9939CQ4P0 grpc-routing %

Note:

Errors are not added for API spec rules.
For RegularExpression path matches, we leave the cookie Path empty. A regex can match anywhere within the URL path (for example, /coffee/[A-Za-z]+/tea), so deriving a concrete cookie path from it would be misleading and could unintentionally restrict which requests send the cookie.

Please focus on (optional): If you any specific areas where you would like reviewers to focus their attention or provide
specific feedback, add them here.

Closes #4231

Checklist

Before creating a PR, run through this checklist and mark each as complete.

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have checked that all unit tests pass after adding my changes
I have updated necessary documentation
I have rebased my branch onto main
I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NONE

salonichf5 · 2025-11-19T14:30:22Z

internal/controller/nginx/config/validation/common.go

+	units := []unit{
+		{"ms", 1},
+		{"s", 1000},
+		{"m", 60 * 1000},
+		{"h", 60 * 60 * 1000},


can switch its order if we want larger units first.

sjberman · 2025-11-19T15:35:41Z

FYI, the release note in the PR description isn't going to be used at all since this is being merged into the feature branch. For the main PR that we merge at the end, we'll want a descriptive release note to discuss the different ways session persistence is supported.

sjberman · 2025-11-19T17:27:51Z

See my comment on the other PR, I think we have to rethink this a bit to support a backend being referenced multiple times, to prevent unintended behavior or blocking desired behavior.

tataruty · 2025-11-25T09:44:46Z

internal/controller/nginx/config/upstreams.go

+		Name:        sp.Name,
+		Expiry:      sp.Expiry,
+		Path:        sp.Path,
+		SessionType: string(sp.SessionType),


wow! this PR is just huge work and so impressing!
I'm just wondering how we get the exact cookie from the
< Set-Cookie: coffee-cookie=2b13cc98f0224fca47885f32115ef236; expires=Tue, 18-Nov-25 22:17:55 GMT; max-age=600; path=/coffee
to fill in
sticky cookie coffee-cookie expires=1440m path=/coffee
Is this mechanism somewhere inside NGINX?
The cookie isn't something automatically created it can be set by user during the very first request and then LB will always check any new request for the same cookie?

This is a great question.

When we configure:

upstream coffee_backend { server backend1; server backend2; sticky cookie coffee-cookie expires=1440m path=/coffee; }

we’re not copying an existing cookie into NGINX – we’re just telling NGINX: use a sticky session cookie named coffee-cookie and set its attributes (expires, path, etc.)

On the first request from a client that doesn’t have this cookie yet, NGINX: Picks a backend using the normal Load balancing algorithm. Generates an cookie token (for example 2b13cc98f0224fca47885f32115ef236) and stores a mapping value → backend and sends a Set-Cookie header like:

Set-Cookie: coffee-cookie=2b13cc98f0224fca47885f32115ef236; ...; path=/coffee

The browser then includes that cookie on subsequent requests: Cookie: coffee-cookie=2b13cc98f0224fca47885f32115ef236 and NGINX Plus uses the cookie value to route the request back to the same upstream server. You never need to copy the cookie value into the config; it’s generated and managed by NGINX itself.

github-project-automation bot added this to NGINX Gateway Fabric Nov 18, 2025

github-project-automation bot moved this to 🆕 New in NGINX Gateway Fabric Nov 18, 2025

nginx-bot bot added the release-notes label Nov 18, 2025

github-actions bot added the enhancement New feature or request label Nov 18, 2025

salonichf5 changed the title ~~Add session persistence support for NGINX Plus users using the sticky cookie directive~~ Do not review: Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 commented Nov 19, 2025

View reviewed changes

salonichf5 force-pushed the feat/plus-session-persistence branch from 622cc92 to ded1099 Compare November 19, 2025 14:47

salonichf5 changed the title ~~Do not review: Add session persistence support for NGINX Plus users using the sticky cookie directive~~ Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 marked this pull request as ready for review November 19, 2025 14:49

salonichf5 requested a review from a team as a code owner November 19, 2025 14:49

nginx-bot bot removed the release-notes label Nov 19, 2025

salonichf5 changed the title ~~Add session persistence support for NGINX Plus users using the sticky cookie directive~~ DNR: Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 force-pushed the feat/session-persistence branch 2 times, most recently from 9863bd1 to 758a284 Compare November 24, 2025 17:50

salonichf5 force-pushed the feat/plus-session-persistence branch from 1b4dc06 to fc5c7cb Compare November 24, 2025 18:13

add session persistence support for NGINX plus users

d44ebda

salonichf5 force-pushed the feat/plus-session-persistence branch 2 times, most recently from edd4d9e to fcad961 Compare November 25, 2025 04:53

salonichf5 changed the title ~~DNR: Add session persistence support for NGINX Plus users using the sticky cookie directive~~ Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 25, 2025

add sp config to backend ref

864ddd8

salonichf5 force-pushed the feat/plus-session-persistence branch from fcad961 to 864ddd8 Compare November 25, 2025 05:00

tataruty reviewed Nov 25, 2025

View reviewed changes

salonichf5 requested a review from a team November 25, 2025 18:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

Uh oh!

salonichf5 commented Nov 18, 2025 •

edited

Loading

Uh oh!

salonichf5 Nov 19, 2025

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

tataruty Nov 25, 2025

Uh oh!

salonichf5 Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add session persistence support for NGINX Plus users using the sticky cookie directive #4305

Are you sure you want to change the base?

Add session persistence support for NGINX Plus users using the sticky cookie directive #4305

Uh oh!

Conversation

salonichf5 commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Checklist

Release notes

Uh oh!

salonichf5 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

tataruty Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

salonichf5 Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

salonichf5 commented Nov 18, 2025 •

edited

Loading